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EXECUTIVE SUMMARY 


Consistent with the longitudinal evaluation of the Intel Teach to the Future Essentials Course that 
Education Development Center’s Center for Children and Technology (CCT) has been conducting 
since the program’s inception in 2000, the Intel Foundation commissioned CCT to conduct a forma- 
tive evaluation of the Leadership Forum Pilot program. This report presents findings and recom- 
mendations from this formative research. 


Methods 


Data collection for this formative research was conducted using mixed methodologies: 


e Survey. EDC developed and advised Intel in the deployment of an online survey that all 
Leadership Forum attendees were asked to complete at the end of each training session. 


e Observations. An EDC evaluator attended and observed six Leadership Forums from September 
through November 2004. 


Key Findings 
Below is a list of key findings from the formative evaluation of the Pilot. 


e From September 27 to November 19, 2004, Intel Innovation in Education held 18 Leadership Forums, 
training 345 administrators, 267 of whom responded to an online survey (77% response rate). 


e Twenty-six percent of survey respondents were district-level administrators (superintendent, curricu- 
lum director, etc.) and 56% were school-level administrators (principal, assistant. principal, tech coor- 
dinator, etc.). There was very little difference in the survey responses submitted by administrators 
working at the district and state levels, suggesting the Forums reached both groups effectively. 


e Nearly all survey respondents (95%) reported the ideas and skills they learned from the Forum either 
will probably (54%) or will definitely (41%) help them improve teacher effectiveness and student 
achievement by supporting and promoting the integration of technology. 


e Almost all of the survey respondents (95%) reported the Forum helped them either to a moderate 
extent (43%) or to a great extent (51%) create a prioritized list of leadership behaviors that impact 
the integration of technology as a tool to improve student learning. Equally high were the number of 
respondents who reported the Forum did the following: examined the critical role educational leaders 
play in the effective integration of technology into teaching and learning; presented an analysis of 
the ISTE NETS-A standards and performance indicators; helped me to develop a personalized action 
plan ready for implementation in my school or district; illustrated concrete strategies that were new 
to me for improving student achievement through the integration of technology into the classroom. 
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e A solid majority of survey respondents found all five of the core modules of the Leadership Forums 
— review of available resources, creation of personal action plan, discussion of case studies and 
leadership behaviors, discussion of best practices and exploration of NETS-A standards — either 
moderately or very useful. 


e Three-quarters of respondents reported feeling either well prepared (16%) or adequately prepared 
(59%) to act on the steps outlined in their Action Plans. 


e The Leadership Forums helped strengthen administrators’ perceptions of and interest in Intel Teach 
to the Future, specifically: 1) 70% of survey respondents were from schools/districts participating 
in Intel Teach to the Future; 2) the Leadership Forums helped strengthen administrators’ percep- 
tions of and interest in Intel Teach to the Future; 3) many participants said they were interested in 
providing additional training for themselves and other administrators; and 4) most of the RTAs and 
trainers interviewed reported they had little or no follow-up planned after the Forums’ completion. 


e The Leadership Forums encountered several implementation challenges: 1) core technologies — 
both Internet connectivity and Turning Point — were sometimes unreliable; 2) the curriculum 
script was too restrained and the format was not interactive enough for some administrators; and 
3) asmall minority of administrators mistakenly expected the Forum to focus on concrete skills 
and tools for them rather than focusing on technology integration at the classroom level. 


Recommendations 


The report contains recommendations based on the findings represented in Sections I and II. 


Continue to fill the void of administrator technology training by expanding the program. 
Administrator responses to this program confirm the program team’s expectation that little profes- 
sional development regarding instructional uses of technology is yet available for administrators. 
Consequently, administrators were very receptive to this program, but also came either with very nar- 
rowly defined goals and expectations (wanting technical training) or with so many needs it was diffi- 
cult for them to focus in on the specific goal-setting activities presented in the current curriculum. 
Although Intel is unlikely to choose to respond to administrators’ interest in skills training, there are 
many opportunities to expand this training to provide further support to administrators who are just 
beginning to learn how to be effective leaders in this area. Possibilities include the following: 


- Partnering with administrator certificate/degree programs offered by schools of education. For 
instance, one of the Forums was embedded in a course administrators were taking at a state uni- 
versity as part of a longer training program. 


- Offering a series of Forums that focus on strategies for implementing Action Plans and opportuni- 
ties for administrators to expand and deepen their goals. Parts of the series — discussion of the 
various Action Plan implementation steps, for instance — could occur online after administrator- 
shad the initial opportunity to train face-to-face. 
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- Building an administrator component into teacher training thereby facilitating administrator- 
teacher communication around technology integration. For instance, the future iteration of 
Teach to the Future could include a Leadership Forum segment, requiring teachers to collaborate 
with principals and other instructional leaders within their schools. 


Continue to use Leadership Forums to strengthen the positive perception of and participa- 
tion in Intel Teach to the Future. While all participants are introduced to Intel's teacher training 
program during the course of the Forums and receive a copy of the “Intel Teach to the Future: 
Meeting the Challenge of No Child Left Behind” booklet, they could absorb more detailed informa- 
tion about how teachers in their schools and districts can pursue additional training opportunities. 
Past evaluation of the Essentials Course has indicated that Intel Teach to the Future often comple- 
ments existing, more basic professional development offerings within school districts. The 
Leadership Forums could include an activity in which administrators conduct an initial needs 
assessment regarding their teachers’ professional development related to educational technology, 
and consider where the Essentials Course might fit within their overall sequence of trainings. More 
concretely, administrators could be given hard copies of information about how to become 
involved in Intel Teach to the Future, and contact information for other districts in their area who 
have already been involved in the program and could provide insight into how to make the best 
use of it. 


Give participants greater opportunity to exchange ideas with one another by building in 
additional paired and group activities. Many participants expressed a desire to hear from their 
colleagues about technology integration within their local settings and to use the Forum as an 
opportunity to develop concrete ways to work together within their schools and districts. In the 
Essentials Course, the extended training period gave teachers time to build these relationships and 
develop plans for future collaboration. This kind of peer learning and collegial support are consis- 
tent with the spirit of all of Intel's professional development programs, and should be emphasized 
in the Leadership Forums as well. However, their brief timeframe means these kinds of activities 
must be structured into the program itself, because there is little or no time for informal discus- 
sion or follow-up conversations among participants. The Leadership Forums should emphasize the 
importance of learning from one another's experiences and the value of future collaboration 
among participants by building time for structured, focused planning around these topics into the 
Forum agenda. 


Reduce the level of “scripting” in the training experience by encouraging facilitators to 
familiarize themselves with the substance of the curriculum rather than relying on specific 
language. As the program expands and the need to recruit trainers grows it will be important to 
communicate clearly what is expected of each trainer and to model these expectations in the 
training they receive. This will be true of both newly recruited trainers as well as administrators 
who initially attend a Forum and then have an interest in training fellow administrators once they 
return to their schools and districts. Although requiring all trainers to read a curricular script 
ensures the delivery of the same content in every Forum, it does not give the trainer the flexibili- 
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ty to adapt to local needs and interests, potentially making the training less valuable to partici- 
pants. Therefore, future training and support materials, such an online resource constructed for 
trainers in the field, may include a simple description of key points worth emphasizing in each 
curriculum module. Another useful resource would be a list of “tried-and-true” adaptations that 
trainers have made depending on various local conditions and particular kinds of participant 
groups. 


Conduct detailed technical run-throughs prior to each training session. Trainers as well as an 
on-site technical support staff will minimize the number of technical delays and glitches by doing 
the following: running Turning Point software checks, conducting software inventories, including 
Flash, preloading relevant Internet sites, and testing the connection between the laptop and pro- 
jector. Ideally, the same tech support person, who is familiar with the facility and other demands 
that are likely to occur on infrastructure at the time of the training, will remain present during 
the training to troubleshoot any difficulties that arise. 


Use the next phase of evaluation to identify concrete ways participants use their training, 
tracking the implementation of their Action Plans and how they support technology integra- 
tion at the classroom level based on their Forum experience. Nearly all of the participants 
interviewed expressed a strong desire to implement their Action Plans but relatively few (16%) 
described themselves as being “well prepared” to do so. Tracking if and how administrators take 
action to accomplish these goals is likely to yield valuable information about the local and sys- 
temic challenges administrators encounter, specific knowledge or information they may need in 
order to act effectively, or different decision-making or policy-setting practices that prove to be 
more or less successful. Learning more about all of these topics will inform further development of 
the training experience and help to define the scope and depth of impact the program can expect 
to have. 


Revise the online survey — or administer two versions — to capture a broader range of 
responses. Just as the Forum curriculum will be updated, so too must the survey to reflect the 
changes and newly refined goals. While this survey provided useful feedback on participants’ 
responses to the Forum, responses were highly consistent across questions and across different 
groups of respondents. It is appropriate to develop a more focused survey at this point that would 
allow for more discrimination among participants’ different experiences with the program. 


Appendices 

The reports contains three appendices: 
e Introductory Letter to Online Survey 
e Online Survey 


e Online Survey Basic Frequencies 
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INTRODUCTION 


uilding on the four-year momentum of the Intel Teach to 
the Future training initiative aimed at classroom teachers, the Intel Leadership Forum is 
a program designed for school- and district-level administrators throughout the United 
States. Recognizing the important role administrators play in supporting teachers’ 
efforts, the training experience was developed to guide K-12 instructional leaders in their efforts 
to plan and implement technology integration and professional development in their schools, 
especially those schools that previously had participated in the teacher training program. 
Specifically, the Forum gave participating administrators the opportunity to do the following: 


e Examine the critical role leaders play in the effective integration of technology into teaching 
and learning 


e Apply their knowledge to create a prioritized list of leadership behaviors that impact the inte- 
gration of technology as a tool to improve student learning 


e Analyze the International Society for Technology in Education (ISTE) National Educational 
Technology Standards for Administrators (NETS-A) 


e Synthesize their understanding to develop a personalized action plan ready for implementation 


Each administrator participating in the Forums received a curriculum guide with a complimentary 
CD-ROM and spent three or four hours engaged in hands-on activities in a computer lab setting. 
Using a PowerPoint presentation as well as print and web-based tools and resources, a trainer (or 
co-trainers) facilitated the Forum curriculum, which consisted of six separate but related modules: 


1) Introducing the Forum 


2) Exploring best practices, including viewing the Intel Innovation in Education Web Site and 
exploring strategies and exemplary unit plans 


3) Examining leadership standards and behaviors, including doing a hands-on activity with the 
“Visual Ranking” tool and reading case studies featuring real-world administrators 


4) Constructing a personal action plan 
5) Touring available resources included on the web and curriculum CD-ROM 
6) Wrapping up 


Consistent with the longitudinal evaluation of the Intel Teach to the Future that Education 
Development Center’s Center for Children and Technology (CCT) has been conducting since the pro- 
gram’s inception in 2000, the Intel Foundation commissioned CCT to conduct a formative evalua- 
tion of the Leadership Forum Pilot program. Two goals guided the 2004 evaluation of the U.S. Intel 
Teach to the Future Leadership Forums: 
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e Understanding how the program is being received by participating administrators and what 
qualities of the program are supporting or impeding the usefulness of the Forums 


e Understanding participants’ perceptions of whether and how the program is likely to influence 
their actions in their own schools and districts and what factors may be impeding or supporting 
their efforts to follow up on the experience. 


This report is comprised of five sections: 

1) Key Findings 

2] Description of Leadership Forum Participants 
3) Detailed Findings 

4] Recommendations 


5) Appendices 


Methodology 


Data collection for this formative research was conducted using mixed methodologies. 


Survey. EDC developed and advised Intel in the deployment of an online survey that all Leadership 
Forum attendees were asked to complete at the end of each training session.’ The survey instru- 
ment investigated the following: 


e Administrators’ perceptions of and responses to the program 


e Their current practices and beliefs regarding the development of their local educational technol- 
ogy program 


e Their goals for further expansion or improvement of those programs 


e Their perceptions of the local conditions that are supporting or impeding their efforts to 
improve the use of educational technology in their schools or districts 


This survey was launched prior to the beginning of the program pilot in July. EDC worked with 
Intel to ensure that program participants were able to complete the online survey at the conclu- 
sion of their Forums. The survey was web-based and made use of Zoomerang technology. 


Observations. An EDC evaluator attended and observed six Leadership Forums from September 
through November 2004. The evaluator worked with Intel staff and Regional Training Agency 
(TRA) contacts to guide site selections, with a goal of visiting a wide range of Forums that varied 
by geographic region and types of districts and participants involved. See Table 1 for an overview 


‘See Appendices 1 and 2 for copy of the online introductory letter and survey. 
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of observed Forums. The evaluator also worked with ICT staff to determine which of the scheduled 
Forums were likely to occur based on early registration. A minimum of eighteen people were 
required to register at least three weeks in advance for each Forum; 10 to 29 administrators partic- 
ipated at each of the six observed Forums. 


During each site visit, the evaluator paid particular attention to the following: 


e Whether and how trainers’ presentation corresponded to the structure and content of the train- 
ing materials, and what local circumstances required the trainer to adapt their approach 


e How key messages about administrators’ roles in supporting the use of educational technology 
were communicated 


e To what extent administrators were engaged by, contributed to, and responded to discussions 
and activities during the Forum 


e Whether and how administrators discussed how the ideas presented in the Forum connected to 
their current local situations and goals, and what local challenges and opportunities they per- 
ceived to be relevant to achieving the goals discussed in the Forum 


Table 1: Overview of Observed Forums 


FORUM 1 FORUM 2 FORUM 3 FORUM 4 FORUM 5 FORUM 6 
Region Southwest Rocky New New Northwest South 
Mountains England England 
Training High School Annual Regional Elementary University Theater within 
Computer Professional PD School Computer Science 
Lab Conference Center Computer Lab Lab Center 
Participants/ All from Multiple All from All from Multiple Multiple 
Districts single rural districts single urban single urban districts districts 
district (rural, urban district district (primarily (rural, urban, 
suburban) across rural) through- suburban) 
the country out one state throughout 
one state 
Length of 4 hours 3 hours 4 hours 4 hours 4 hours 4 hours 
Session 
Trainer ICT Staff ITC Staff Curriculum Curriculum University Local 
and Local Co-author Co-author Professor Principal 
Superintendent 
Number of 19 17 12 17 29 10 
Participants 


During these visits the evaluator also conducted informal interviews with the trainer immediately 
following the Forum or established a later time to conduct an interview with him or her via tele- 
phone. These interviews served a two-fold purpose: 1) to document the trainer’s perceptions of the 
goals and core concepts of the Forum and 2) to gather his or her views of how successful this par- 
ticular Forum experience was and how interested in and ready for the material administrators 
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were. When possible, just after the conclusion of the Forum or during breaks, the evaluator also 
conducted informal interviews with participating administrators to explore their perceptions of the 
goals, strengths and weaknesses of the Forum. The evaluator probed both for administrators’ inter- 
pretations of the relevance and utility of the Forum relative to their own needs and current prac- 
tices and resources and for their level of engagement with and understanding of the core concepts 
the Forum was intended to introduce. 


Phone interviews. The EDC evaluator conducted 12 phone interviews and email exchanges with 
administrators who participated in Forums, most of which were different from the ones the evalua- 
tor observed. These phone interviews and electronic exchanges investigated topics similar to the 
informal interviews described above and explored in more depth how administrators may be plan- 
ning to make use of key concepts from the Forum and apply them to their local circumstances. 
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SECTION I. DESCRIPTION OF LEADERSHIP FORUM 
PARTICIPANTS 


From September 27 through November 19, 2004, Intel Innovation in Education held 18 Leadership 
Forums throughout the country. Three hundred forty-five administrators attended these Forums, 267 of 
which submitted responses to the online evaluation (77% response rate). Based on these responses, 
below is a profile of the participants and the characteristics that defined them. 


Participants’ Roles and Experience 


Participants held a variety of administrative positions within their schools and districts. The largest 
grouping (39%) was principals, followed by assistant principals (11%) and district-level curriculum direc- 
tors, coordinators, or supervisors (10%). For a complete breakdown of participant's positions, see Table 2. 


Table 2: Participants’ Job Titles 


NUMBER OF STUDENTS ENROLLED IN DISTRICT PERCENTAGE OF INTEL LEADERSHIP 
FORUM PARTICIPANTS (N=266) 
District superintendent 5% 
District assistant superintendent 3% 
District-level curriculum director, coordinator, or supervisor 10% 
District-level technology director, coordinator, or supervisor 8% 
School principal 39% 
School assistant principal 11% 
School-level curriculum director, coordinator, or supervisor 3% 
School-level technology director, coordinator, or supervisor 3% 
Classroom teacher or other instructional staff 8% 
Other* 11% 


* Participants who indicated “other” specified a wide range of job titles. Examples include: Special Education Director, District 
Technology Resource Teacher, Media Director, Director of Research and Development and School Improvement Specialist. 


On average the participants reported having nearly 13 years of experience teaching: 7 years of experi- 
ence as a principal and 6 years of experience as a K-12 administrator. The participating principals had an 
average of 7 years of experience with few participants having more than two decades of experience. See 
Table 3 for a comparison with national percentages. 


Table 3: Principals’ Years of Experience 


NUMBER OF YEARS OF EXPERIENCE PERCENTAGE OF INTEL LEADERSHIP NATIONAL 
AS A PRINCIPAL FORUM PARTICIPANTS PERCENTAGE 
3 or fewer 36% 30% 
4-9 38% 30% 
10-19 23% 28% 


20 or more 4% 13% 
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Demographic Information 


Participants represented a range of school districts, in terms of student populations, size of the 
district, and student eligibility for free and reduced lunch. 


Thirty percent of participants worked in school districts that had 25,000-99,999 students, which is 
much higher than the national percentage, and nearly half worked in districts that had fewer than 
7,500 students. For a complete accounting of district student enrollment and comparisons to 
national percentages, see Table 4. 


Table 4: District Student Enrollment 


NUMBER OF STUDENTS PERCENTAGE OF INTEL LEADERSHIP NATIONAL 
ENROLLED IN DISTRICT FORUM PARTICIPANTS PERCENTAGE 
1 - 299 6% 22% 
300 - 999 12% 27% 
1,000 - 4,999 25% 38% 
5,000 - 7,499 13% 5% 
7,500 - 9,999 5% 2% 
10,000 - 24,999 7% 4% 
25,000 - 99,999 30% 2% 
100,000 or more 1% <1% 
Not sure 1% 


Approximately half of the participants (53%) worked in districts comprised of ten or fewer schools 
and 23% worked in districts of 21 to 50 schools. For a complete accounting of number of schools 
in the district, see Table 5. 


Table 5: Number of Schools in District 


NUMBER OF SCHOOLS IN PERCENTAGE OF INTEL LEADERSHIP 
DISTRICT FORUM PARTICIPANTS 

1-5 29% 

6-10 24% 

11 - 20 6% 

21 - 50 23% 

51 - 100 12% 

More than 100 5% 

Not sure 1% 


Approximately three-fourths of participants (73%) worked primarily within a single school rather 
than an entire district. Therefore, we asked participants to indicate the size of their schools. 


EDC| CENTER FOR CHILDREN & TECHNOLOGY 
7 


Thirteen percent worked in schools with fewer than 300 students compared to 33% nationally and 
39% worked in schools with 300 to 749 students compared to 47%. For a complete accounting of 
the number of students in participants’ schools, see Table 6. 


Table 6: Number of Students in School 


NUMBER OF STUDENTS PERCENTAGE OF INTEL LEADERSHIP NATIONAL 
IN SCHOOL FORUM PARTICIPANTS PERCENTAGE 
1-99 2% 11% 
100 - 299 11% 21% 
300 - 749 39% 47% 
750 - 1,499 16% 16% 
1,500 or above 4% 4% 

Not applicable 27% 


The socioeconomic composition of participants’ districts, as indicated by the number of students 
qualifying for free or reduced-price lunch, was evenly split: about one third worked in districts 
where half or fewer students were eligible; roughly one-third represented districts where three- 
quarters to half of students were eligible; and roughly one-third came from districts where all to 
three-quarters of there students qualified. For a complete accounting of district and school eligi- 
bility for free or reduced-price lunch, see Table 7. 


Table 7: District and School Eligibility for Free or Reduced-Price Lunch 


PERCENTAGE PERCENTAGE OF INTEL PERCENTAGE OF INTEL NATIONAL PERCENTAGE 
ELIGIBLE FOR LEADERSHIP FORUM LEADERSHIP FORUM BY SCHOOL 
F/R LUNCH PARTICIPANTS - BY DISTRICT PARTICIPANTS - BY SCHOOL 
(N=260) (N=251) 
0 - 25% 10% 15% 34% 
26 - 50% 23% 18% 34% 
51 - 75% 31% 19% 16% 
76 - 100% 30% 22% 16% 
Not sure 6% <1% 


Not applicable 25% 
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SECTION II: DETAILED FINDINGS 


The findings are grouped into five categories: overall training experience; action plans and other 
curricular modules; Intel Teach to the Future and other future plans; training materials; and facili- 
tation and implementation challenges. While none of these categories is mutually exclusive each 
one relates to an important layer of the Leadership Forum. Furthermore, although the online sur- 
vey asked participants to designate their current position, e.g. principal or superintendent, there 
was no difference in any of the responses they gave to questions in the survey between those 
working at the district level and those working at the school level, suggesting the Forums reached 
these two groups equally. As a result, there are no distinctions between these two domains of 
responsibility in the reporting of the findings below. 


Overall Training Experience 


Overall, participants reported they were satisfied with their training experience and valued 
what they learned. The vast majority of survey respondents (93%) said they either probably 
(45%) or definitely (49%) would recommend the Leadership Forum to a colleague. As one superin- 
tendent from a district in the Southwest summarized, “It was excellent. It was very interactive...I 
thought the training was outstanding.” Similarly, 92% of respondents reported they agreed (44%) 
or strongly agreed (49%) the goals and objectives of the Forum were clearly stated and the agenda 
was followed (n=264). Ninety-four percent of respondents said the Forum examined the critical 
role educational leaders play in the effective integration of technology into teaching and learning 
to a moderate or great extent and 88% said it helped them create a prioritized list of leadership 
behaviors that impact the integration of technology as a tool to improve student learning (n=266). 
See Figure 1 for a full summary of responses to this question. 
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Figure 1: Participants’ Statements About the Forum Experience 
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At the conclusion of the Forum, participants reported feeling strongly about three of the 
four primary areas addressed during the training and approximately half said their opin- 
ions either had slightly or significantly changed as a result of their training experience. 

Survey respondents said they strongly agreed with the following statements: 


e Administrative leadership plays an important role in ensuring the success of classroom technol- 
ogy integration (81%, n=265). 


e It is important for administrative leaders to model the use of technology for teaching and learn- 
ing within their school or district (79%, n=265). 


e Technology is an important tool that should be used to enhance teacher effectiveness and stu- 
dent achievement (79%, n=266). 


Furthermore, although roughly half of the respondents said their opinions about leaders’ roles and 
leadership behaviors did not change as a result of their training, the other half reported their 
opinions changed slightly or significantly because of their participation in the Forum. There was 
no correlation between the responses participants had given to these two questions. Respondents 
who rated one of the statements within Question 12 highly rated all the other Question 12 state- 
ments highly. This was also true of Question 13. Nevertheless, the degree of opinion change was 
not related to their level of agreement with each statement. 
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At the conclusion of the Forum, a majority of participants reported they felt familiar with 
the NETS-A standards, though a portion said they were not familiar with them. Sixty percent 
of survey respondents said they agreed with the statement, “I am familiar with the NETS-A stan- 
dards for administrators’ use of technology” though 17% disagreed and 3% strongly disagreed with 
it (n=264). Significantly, 83% of respondents said their familiarity with the standards had changed 
slightly or significantly as a result of their participation in the Forum (n= 261). Similarly, 85% of 
respondents reported the Forum had presented an analysis of the ISTE NETS-A standards and per- 
formance indicators to a moderate or great extent (n=266). One trainer attributed this to the con- 
nection the curriculum drew between the standards and the personal action plans. He said, “We 
asked people to fill out three parts of the [Action Plan] matrix. Some people filled out every sin- 
gle block. That showed me that we met a need: people are trying to fit the national standards to 
their own environments.... That was a culminating validation. We wanted leaders to take responsi- 
bility of integrating technology in their schools, and it worked.” 


Exchanges among participants — and between participants and trainers — were where a 
great deal of the learning took place. Administrators reported they valued having the opportu- 
nity to spend time with their peers discussing their various approaches to technology integration. 
Some participants said they would have liked additional time to converse with one another built 
into their training session. By contrast, several RTAs said they were eager for the Forums to keep 
to their timeline and said they were concerned that administrators were “talking too much.” 
Nevertheless, 87% of survey respondents reported they agreed or strongly agreed they had been 
engaged in group discussions on the topics presented in the Forum (n=265). 


Action Plans and Other Curricular Modules 


Nearly all participants reported they would be better able to support technology integration 
among their teaching staff as a result of their training. Ninety-five percent of survey respondents 
said the ideas and skills they learned from the Forum either would probably (54%) or definitely 
(41%) help them improve teacher effectiveness and student achievement by supporting and pro- 
moting the integration of technology (n=273). See Figure 2 for a full summary of responses to this 
question. 
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Figure 2: Likelihood of Forum Ideas and Skills Improving Teacher Effectiveness and Student 
Achievement by Supporting Technology 


54% Probably Yes 


41% Definitely Yes 


0% Definitely Not 


5% Probably Not 


The majority of participants said the Action Plan they had created during the Forum would 
help them implement their technology-related goals. Among survey respondents, 59% reported 
feeling adequately prepared and 16% well prepared to act on the steps they had outlined in the 
Action Plans (n=271). See Figure 3 for a summary of responses. Likewise the majority of respon- 
dents said the Forum had to a great extent (30%) or moderate extent (50%) helped them develop 
a personalized action plan ready for implementation in their school or district (n=266). One train- 
er attributed the success of the Action Plan to the Visual Ranking activity. He explained, “It gives 
everyone a starting point to get to the action plan. They see the priorities depending on the role 
they play — teacher, administrator. It’s a way to have content drive the technology.” 


Figure 3: Participants’ Level of Preparation to Implement Action Plans 


16% Well Prepared 


3% Unprepared 


22% Somewhat Prepared 


59% Adequately Prepared 
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Participants identified all factors identified in the survey as strong “driving forces” in their 
efforts to achieve their Action Plans. Survey respondents said the following would help support 
or sustain their process of change: stated school/district goals for technology integration (62%); 
interests and priorities of other administrators in my school/district (57%); the technology 
resources (equipment) available in my school/district (55%); interests and priorities of teachers in 
my school/ district (56%); Federal/State mandates related to technology integration (52%); scope 
and focus of budget allocations associated with technology integration within my district (49%); 
and interests and priorities of the school board in my district (47%). 


The majority of participants said they walked away from the training with new ideas for 
integrating technology into their teaching and learning goals. Among the survey respondents, 
73% reported the Forum illustrated concrete strategies that were new to them for improving stu- 
dent achievement through the integration of technology into the classroom, either to a moderate 
or great extent (n=265). Several participants reported during the interviews that they looked for- 
ward to finding the time to explore the resources and other materials they had received during the 
training while others said they returned to their schools and immediately shared the resources 
with fellow administrators and/or classroom teachers. Nearly all of the administrators interviewed 
said it was too soon to know how effective their new skills would be — or if they would have time 
to make use of them. 


A clear majority of participants reported finding each of the substantive modules useful. 
Consistently, approximately 80% of survey respondents said the five core content areas — explo- 
ration of NETS-A standards, discussion of best practices, discussion of a case study and leader 
behaviors, creation of a personal action plan, and review of available resources — were either 
moderately or very useful. Equally consistent, only one-fifth of participants said these five areas 
were somewhat or not useful. In interviews, both trainers and administrators singled out Visual 
Ranking, the lesson plans found on the Intel Innovation in Education website, the NETS-A stan- 
dards, and the resources to be among the materials and tools they valued the most. See Figure 4 
for participants’ response to the core curricular modules. A few people also expressed specific 
opinions about the merits and utility of the Case Studies. For example, one trainer said about the 
Case Studies: “They take more time than they are worth. I don’t think the administrators think 
reading those at the time of training are a good use of time.” Alternatively, one principal reported 
during an interview the Case Studies could serve as a starting point for the training. She said, 
“The case studies are perfect. I thought we'd start there and then say, ‘Look principals this is how 
your day could look’ This guy [the principal in the Case Study] is click, click, click and our day is 
write, write, write. I thought we’d ask: what components could you use in your world. Why does- 
n't your world fit with this? Are there tools you don't have? Are there things you don’t know? How 
do you build your technology plan from there? You could compare yourself to the ‘technology 
principal’ and move from there. That’s the way everyone was relating to this material.” 
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Figure 4: Usefulness of the Curriculum Modules 
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Intel Teach to the Future and Other Future Plans 


The Leadership Forums helped strengthen administrators’ perceptions of and interest in 
Intel Teach to the Future. In discussing their recruitment strategy for the Forums, several RTAs 
interviewed said they sought out administrators already familiar with Intel Innovation in 
Education's teacher training program. It is not surprising therefore that 70% of survey respondents 
were tied to schools or districts that had participated. For example, one professional development 
coordinator who helped coordinate one of the Forums said of the 190 staff in his district, 130 had 
gone through the 40 hours of Intel Teach to the Future training. Despite their general familiarity, 
several administrators reported during interviews the Forum helped them gain a better under- 
standing of the kinds of issues surrounding technology integration. They also said they had an 
increased interest in seeing teacher training expand and/or said they had acquired an improved 
view of the support Intel Innovation in Education provides practitioners. Other administrators 
interviewed said they returned from the Forum ready to encourage their teachers to get involved 
in the Teach to the Future program. As one principal explained, “I thought it was excellent. I 
came back and touted the program to my teachers. This [Teach to the Future] was offered at my 
last school but we didn’t pursue it. Now I’m happy to be somewhere where we can get involved. I 
was very impressed.” 
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Many participants said they were interested in providing additional training for themselves 
and other administrators. Participants said they would like to identify ways to train other 
administrators either by having them attend a formal Intel Leadership Forum or by recreating the 
experience themselves in a staff development meeting where they could tailor it to local needs. 
One trainer reported this would be possible because “the curriculum was self-standing.” While 
some participants said they definitely would pursue additional training through Intel Innovation 
in Education, others said they were awaiting grants and other sources of support to determine 
what would be possible given potential budget constraints. Two RTAs reported during interviews 
how they would continue to try to take advantage of “natural opportunities,” such as monthly 
principals’ meetings and other already scheduled events, to offer additional training. As one train- 
er who had arranged to offer the Forum as a part of one course in a two-year Superintendent 
Endorsement program explained, “Figuring out how to get people to events is a real challenge. 
Standards and a budget crunch make that hard. We try to capitalize on events that are already 
happening, where people are already gathering. People are too busy to come to something that is 
separate. The embedded way is the only way to achieve success.” 


Most of the RTAs and trainers interviewed reported they had little or no follow-up planned after 
the Forums’ completion. Several trainers said they did not have a specific strategy for contacting 
participating administrators but, as one trainer commented, “We don't have a next step but if we 
did, there is a lot of good will so the next step would be an easy one to take.” And among those 
that did have a plan following the Forums, the emphasis would shift toward teacher training. As 
one RTA stated, “I think follow-up [with administrators] doesn’t really makes sense because they 
are busy people, dealing with one crisis after the next. The focus is on Teach.... ideally the princi- 
pals [who were trained] would send 15 teachers to get trained.” 


Training Materials 


The guide and CD-ROM were clear and easy-to-follow as was the overall training. Ninety-two 
percent of survey respondents said they agreed (45%) or strongly agreed (47%) the directions were 
clear for each activity (n=262). Likewise, 90% reported the session was organized and the materi- 
als were helpful (n=263). One trainer explained, “The format of the manual is very well organized. 
It is very strong. You can go back and go through it at your own station and again when you have 
the time. And the manual is very step-by-step so that makes it easy.” Similarly, during interviews 
several administrators said they appreciated receiving both the guide and the CD-ROM. During an 
interview, one trainer offered a counterpoint, saying, “The confusion about what is on the CD and 
what is on the web is huge. Upfront, you want to make this clear. Giving a little more clarity 
about the two would help people who are less comfortable switching through windows. We should 
help trainers understand the two better during training.” 


The Turning Point technology was an effective tool for generating administrators’ participa- 
tion and enthusiasm. Most of the administrators, trainers and RTAs interviewed said they enjoyed 
interacting with the audience response system. However, one RTA said that using Turning Point as 
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a give-away incentive could be reconsidered. He said, “I'd like to see some kind of technology 
incentive rather than one door prize, even if you have to decrease the numbers. Make a special 
incentive to get there. “ In a previous training program funded through a grant he said all partici- 
pants receive a laptop. 


Participants appreciated receiving flash drives and leather-bound journals and the trainers 
enjoyed distributing the materials. As one trainer described, “It’s a classy touch giving the par- 
ticipants the fobs (drives) and the certificates. It is good of Intel to do that. It’s also nice to walk 
out with a guide and the disc, both of which are comprehensive. The administrators could give a 
mini-in-service on their own with the materials they receive.” Half of the trainers observed by the 
evaluator made a point of distributing the flash drives and describing how administrators could use 
them to save their Action Plans. They explained this hands-on walk-through increased the value of 
the drives because administrators who were unfamiliar with the technology would know how to actu- 
ally use it when they returned to their buildings. As one trainer explained during an interview, “I 
make the flash drive part of the experience. Otherwise, people who didn't know how to use the drive 
didn’t use it, they give it to someone else. I have them pull up their action plan and save it. That is 
one of those psychologically powerful experiences. They no longer worry about ruining the machine 
because they haven't used it right.” 


Trainers appreciated Intel Innovation in Education’s commitment to improving the Leadership 
Forum materials and some had identified changes they would like to see made. As one trainer 
explained, “I was amazed by the additions to the slides since the original curriculum was developed. 
Intel is geared up to make changes up until the last. We're encouraged to download the latest 
PowerPoint right before each training.” 


Trainers reported the four-hour version of the Forum was much more effective than the three- 
hour version. While acknowledging how little time administrators have outside of their offices and 
daily commitments, nearly every trainer said they could accomplish a great deal more in the longer 
training session than in the shorter one. Specifically, they cited the opportunity to engage partici- 
pants in more discussion and the chance to use the Turning Point software to ask additional questions 
about participants’ response to specific modules. 


Facilitation and Implementation Challenges 


Overall, participants reported they were impressed by the trainer's ability to create a worth- 
while experience. Among the survey respondents, 55% said the trainer was very effective at facilitat- 
ing the training and another 36% said the trainer was adequately effective (n=264).’ 


The trainer was responsible for setting the tone and maintaining learning throughout the 
Forum. By design, the content of a Forum is delivered through a trainer and the resources 
(PowerPoint slides, Intel Innovation in Education website, Turning Point questions, guide, CD-ROM, 
etc.) available to him or her. As one trainer summarized, “We have been told to stick with the script 
and to try and keep it as similar as possible.” And, overall, each trainer observed succeeded in follow- 


* See Appendix 3 for a listing of all basic frequencies from the online survey. 
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ing the curriculum and the sequence of the modules. There were, however, several key points in the 
training where the trainer's individual style and understanding became apparent: 


e Technology glitches: When the technology failed there was an even greater focus on the presenter 
and his or her ability to keep the training moving. One trainer suggested having a hands-on back- 
up plan for the Visual Ranking activity. He thought a stack of cards, each representing a standard 
would allow participants the same learning opportunity albeit in a low-tech way. 


e Balance of screen time and reading time. Trainers varied in how much of participants’ attention they 
drew to their individual monitors, the PowerPoint slides projected on a large screen at the front of 
the room, the printed curricular guides and themselves. Some verbally encouraged individual learn- 
ing styles while others were more prescriptive about where participants were to focus. 


Several participants and trainers said the curricular script was too restrained and the format 
was not interactive. Both trainers and participants alike reported during interviews the script the 
trainers followed, which one facilitator called “highly prescriptive” was too rigid, leaving little room 
for genuine interaction. As one survey respondent commented, “Our trainer was fantastic, but I do not 
like being read to. I would have preferred to have someone who could do the training without having 
to read it from the book. Truthfully, I didn’t stay until the end because of this!” One of the trainers 
had a similar perspective and explained during an interview, “It is impossible for me to train with the 
kind of energy and response to the moment that is necessary. I hit all the content but not with a 
script. I think that’s a matter of personal style — I like a more fluid experience. It validates the peo- 
ple in front of us.... If I’m reading a script I think the participants disengage more.” And an RTA, who 
suggested adding opportunities for participants to “get out of their seats and engage in hands-on 
activities” said, “The problem is Intel is running this like a corporate training. And we're educators.” 


Technology was sometimes unreliable. A few trainers explained they had requested a technical sup- 
port person to be on-site during the training to troubleshoot and resolve any technical difficulties 
while others said they tested the server and Turning Point software before their Forums began. 
Nevertheless, half of the Forums observed had significant challenges with either the Internet connec- 
tion or the Turning Point software not functioning. Several trainers and administrators alike said there 
was a need for specific strategies for dealing with the inevitable technological delays and challenges, 
eliminating the potential drag these unexpected glitches had on the curriculum. 


The Forum setting had to accommodate the logistical needs of the participants. Given the hectic lives of 
administrators and the myriad demands on their time, invariably a few participants arrived late while 
others had to move in and out of the room to field phone calls from their schools and offices. 


For a small minority of administrators, the Forums were inaccurately portrayed as “administra- 
tor training.” Several participants reported during interviews they had the mistaken impression the 
Forum would give them concrete skills as administrators rather than focusing on teachers’ integration 
of technology. As one trainer stated during an interview, “The emphasis is on teachers, not principals. 
For some, this was a missed opportunity.” Similarly, a principal explained during a Forum break she 
thought the training would introduce her to materials and resources she could use to be a better 
leader, such as technology tools for supervising teachers. She said, “We want to know how to value 
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technology in our lives.... You have to get leaders — us — to see what the value is and how to use 
it. For example, we want to see how it can diversify and differentiate our teachers’ instruction. Then, 
it will feed into teachers’ practice. If you are a ‘technology principal’ then you are going to have a 
technology school. ... I was doing an observation the other day and would have loved to have seen 
standards on a handheld that had check boxes with those standards.” For a few other participants, 
because they saw the emphasis being on teachers they believed the Forum could have been much 
shorter. As a director of educational services commented in an interview, “Everything was fine, but 
the administrators’ didn’t need the depth... the administrators resented the time. This was no 
fault of the trainers. The administrators needed to get back to their buildings and they wanted 
just a brief overview [of the teacher tools].” 


EDC| CENTER FOR CHILDREN & TECHNOLOGY 
18 


SECTION III: RECOMMENDATIONS 


Based on the findings described in Sections I and II, below is a set of recommendations the Intel 
Innovation in Education staff may wish to consider as they further develop the Leadership Forums. 
Several of these suggestions pertain to revisions to the actual training and the larger program 
implementation while others deal more directly with the next phase of evaluation. 


Continue to fill the void of administrator technology training by expanding the program. 
Administrator responses to this program confirm the program team’s expectation that little profes- 
sional development regarding instructional uses of technology is yet available for administrators. 
Consequently, administrators were very receptive to this program, but also came either with very 
narrowly defined goals and expectations (wanting technical training) or with so many needs it was 
difficult for them to focus in on the specific goal-setting activities presented in the current cur- 
riculum. Although Intel is unlikely to choose to respond to administrators’ interest in skills train- 
ing, there are many opportunities to expand this training to provide further support to adminis- 
trators who are just beginning to learn how to be effective leaders in this area. Possibilities 
include the following: 


- Partnering with administrator certificate/degree programs offered by schools of education. For 
instance, one of the Forums was embedded in a course administrators were taking at a state 
university as part of a longer training program. 


- Offering a series of Forums that focus on strategies for implementing Action Plans and opportu- 
nities for administrators to expand and deepen their goals. Parts of the series — discussion of 
the various Action Plan implementation steps, for instance — could occur online after adminis- 
trators had the initial opportunity to train face-to-face. 


- Building an administrator component into teacher training thereby facilitating administrator- 
teacher communication around technology integration. For instance, the future iteration of 
Teach to the Future could include a Leadership Forum segment, requiring teachers to collaborate 
with principals and other instructional leaders within their schools. 


Continue to use Leadership Forums to strengthen the positive perception of and participa- 
tion in Intel Teach to the Future. While all participants are introduced to Intel's teacher training 
program during the course of the Forums and receive a copy of the “Intel Teach to the Future: 
Meeting the Challenge of No Child Left Behind” booklet, they could absorb more detailed informa- 
tion about how teachers in their schools and districts can pursue additional training opportunities. 
Past evaluation of the Essentials Course has indicated that Intel Teach to the Future often comple- 
ments existing, more basic professional development offerings within school districts. The 
Leadership Forums could include an activity in which administrators conduct an initial needs 
assessment regarding their teachers’ professional development related to educational technology, 
and consider where the Essentials Course might fit within their overall sequence of trainings. More 
concretely, administrators could be given hard copies of information about how to become 
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involved in Intel Teach to the Future, and contact information for other districts in their area who 
have already been involved in the program and could provide insight into how to make the best 
use of it. 


Give participants greater opportunity to exchange ideas with one another by building in 
additional paired and group activities. Many participants expressed a desire to hear from their 
colleagues about technology integration within their local settings and to use the Forum as an 
opportunity to develop concrete ways to work together within their schools and districts. In the 
Essentials Course, the extended training period gave teachers time to build these relationships and 
develop plans for future collaboration. This kind of peer learning and collegial support are consis- 
tent with the spirit of all of Intel's professional development programs, and should be emphasized 
in the Leadership Forums as well. However, their brief timeframe means these kinds of activities 
must be structured into the program itself, because there is little or no time for informal discus- 
sion or follow-up conversations among participants. The Leadership Forums should emphasize the 
importance of learning from one another's experiences and the value of future collaboration 
among participants by building time for structured, focused planning around these topics into the 
Forum agenda. 


Reduce the level of “scripting” in the training experience by encouraging facilitators to 
familiarize themselves with the substance of the curriculum rather than relying on specific 
language. As the program expands and the need to recruit trainers grows it will be important to 
communicate clearly what is expected of each trainer and to model these expectations in the 
training they receive. This will be true of both newly recruited trainers as well as administrators 
who initially attend a Forum and then have an interest in training fellow administrators once they 
return to their schools and districts. Although requiring all trainers to read a curricular script 
ensures the delivery of the same content in every Forum, it does not give the trainer the flexibili- 
ty to adapt to local needs and interests, potentially making the training less valuable to partici- 
pants. Therefore, future training and support materials, such an online resource constructed for 
trainers in the field, may include a simple description of key points worth emphasizing in each 
curriculum module. Another useful resource would be a list of “tried-and-true” adaptations that 
trainers have made depending on various local conditions and particular kinds of participant 
groups. 


Conduct detailed technical run-throughs prior to each training session. Trainers as well as an 
on-site technical support staff will minimize the number of technical delays and glitches by doing 
the following: running Turning Point software checks, conducting software inventories, including 
Flash, preloading relevant Internet sites, and testing the connection between the laptop and pro- 
jector. Ideally, the same tech support person, who is familiar with the facility and other demands 
that are likely to occur on infrastructure at the time of the training, will remain present during 
the training to troubleshoot any difficulties that arise. 
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Use the next phase of evaluation to identify concrete ways participants use their training, 
tracking the implementation of their Action Plans and how they support technology integra- 
tion at the classroom level based on their Forum experience. Nearly all of the participants 
interviewed expressed a strong desire to implement their Action Plans but relatively few (16%) 
described themselves as being “well prepared” to do so. Tracking if and how administrators take 
action to accomplish these goals is likely to yield valuable information about the local and sys- 
temic challenges administrators encounter, specific knowledge or information they may need in 
order to act effectively, or different decision-making or policy-setting practices that prove to be 
more or less successful. Learning more about all of these topics will inform further development of 
the training experience and help to define the scope and depth of impact the program can expect 
to have. 


Revise the online survey — or administer two versions — to capture a broader range of 
responses. Just as the Forum curriculum will be updated, so too must the survey to reflect the 
changes and newly refined goals. While this survey provided useful feedback on participants’ 
responses to the Forum, responses were highly consistent across questions and across different 
groups of respondents. It is appropriate to develop a more focused survey at this point that would 
allow for more discrimination among participants’ different experiences with the program. 


